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REMARKS 

In view of the above amendments and following remarks, favorable reconsideration in 
this application is respectfully requested. 

Telephone Interview 

A telephone interview was conducted with Examiner Nguyen on April 22, 2010. The 
invention was discussed, as well as the prior art to Rose. The present Amendment revises the 
claims to overcome the prior art as discussed during the telephone interview, as indicated more 
fully below. 

Rejection of Claims - 35 U.S.C. §103 

The Examiner rejects claims 1, 8, 10, 12-20, 22-31, 36-38 and 44 under 35 U.S.C. §103 
as being unpatentable over Rose (Annotating Real -World Objects Using Augmented Reality). 
Claims 1-43 and 45-47 have been cancelled, claim 44 has been amended, and new claims 48-51 
submitted. 

Claims 44 and 49 require that the objects to be annotated are selected from the captured 

image. In contrast, Rose provides a system which is used to annotate a real-life object. The user 

points a pointing device 6D (shown in Figs. 3.1 and 7.1) at the real-life object. The system 

determines the coordinates for the pointing device, and in response displays the associated 
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annotation ("exhaust manifold" in Fig. 7.1) on the transparent goggles. (See page 11, section 7.) 
Rose does not indicate how the annotations are assigned, but only notes that various geometric 
coordinates are associated with an annotation and an attachment point. (See page 9.) 
Presumably, Rose must use the pointing device 6D in advance to assign an annotation to each 
desired particular geometric coordinate. 

Consequently, this requires a complicated and expensive pointer, such as a six degrees of 
freedom magnetic tracker able to sense the position and orientation of the pointer in space. 
Moreover, the solution disclosed by Rose is only applicable when the user has access to the 
physical object; it is not possible when this object is far away (such as a mountain in a landscape) 
or when the image or video was previously recorded and the physical object is not available 
anymore. 

Further, it is noted that Rose would not be combinable with Lynde (previously applied by 
the Examiner), which uses goggles. Lynde does not teach a system where a pointing device can 
be manipulated by the user in order to select the objects to annotate, and does not teach selecting 
a captured image as claimed. 

Claim 50 requires receiving a radio signal sent by a radio sender of the element. Support 
for claim 50 can be found at paragraph [0041], which explicitly states that "radio input provides 
information concerning their location, as well as meta-information for annotation." Thus, the 
radio signal emitted by a beacon is used to determine the position of this beacon and the 
associated element. The signal can also indicate meta-information associated with the element, 
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and indicate the text that will be annotated. On the other hand, Rose merely suggests a radio 
signal sent by the 6D pointer, but not a radio signal sent by a beacon or other radio-emitter 
attached to each object to annotate. In fact, a radio-emitter could not be attached to each 
annotated part of the engine that Rose wants to annotate. 

Claim 51 requires identification of the annotated object based on the pointing direction of 
the camera. Support is found at paragraphs [0045] and [0046], which states that "a camera 
acquiring image can be aware of its position and shooting direction" and "image/video seen by 
the user can be annotated by information provided from the base station and based on camera 
position and shooting direction." And, Figure 9 shows the shooting direction of the device held 
by the user relates to the elements in the image that are to be annotated. This feature requires for 
example a camera provided with a compass in order to determine its pointing direction. 

Rose only teaches identification based on a 6D pointer which is not part of the camera, 
and which is usually pointing at another direction than the camera, as clearly shown on Fig. 3.1 
for example. Thus Rose necessitates an additional equipment whose position and orientation are 
determined, and does not deliver the same result. Moreover, Rose requires precise calibration 
between the video camera and the pointing device. In fact Rose even describes calibration as an 
essential component of augmented reality (paragraph 4); indeed, calibration is of utmost 
importance in order to map the position and direction of the pointer in the physical world with 
the image taken by the video camera. Such a precise calibration seems only possible in a fixed 
setting, for example in order to annotate an engine placed at a known, fixed location, or using a 
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calibration pattern as shown on Fig. 4.1 of Rose. Such a calibration between pointer and camera 
is not possible, or at least much more difficult, when the video camera is moving and when one 
does not known in advance its position and pointing direction. In this case, an external 6D 
pointing device is of no practical use since the annotating system can not easily determine which 
element of the image is pointed. 

The invention disclosed in the present application concerns a method where the shooting 
direction of the camera itself is determined. It is therefore not needed to align or calibrate this 
direction with the image of the view. This results in a more simple method and system, and 
which completely solves the problems and errors of the calibration process. 

Claims 48 and 51 require annotation of images taken by a mobile phone. None of the 
prior art teaches a mobile phone with camera being known at the filing date of the invention, or 
that phones were provided with sufficient processing power sufficient for processing images or 
computer vision. Consequently, it seems very unlikely that the one skilled in the art would even 
consider to replace the video camera of Rose with a mobile phone. A mobile phone is of no use 
in that system, and the one skilled in the art has no reason to employ a mobile phone in the 
system of Rose, even if mobile phones with a video camera were known at the filing date. It 
seems that the Rose system will only work with a fixed camera, due to the necessity to calibrate 
this camera with the pointer device and with the annotated object. The one skilled in the art will 
not consider a mobile phone with camera for an application and system where a fixed video 
camera appears to be required. Absent any teaching in the prior art to use a mobile phone, the 
Page 8 of 10 

1 23593 .00 1 06/36004594v. 1 



Serial No.: 09/817,314 

Atty. Docket No.: 123593-00106 

Reply to Office Action of December 31, 2009 

Examiner lacks support to contend that the use of a mobile phone would have been obvious to 
one of ordinary skill in the art. 

It appears that automated annotation of views taken by mobile phones is only interesting 
for purposes completely different from the purposes described in the prior art. Rose describes an 
annotating system intended for annotating video images taken by a fixed video camera at a 
known location. This teaches against the use of a mobile phone with camera, which is poorly 
adapted to this purpose and would not be chosen by the skilled person. A mobile phone camera is 
only useful for annotating images taken on the go, such as landscape images or video of people 
in a meeting. Such an application of an annotating system has not, however, be considered or 
disclosed in any of the cited documents. 

In the event there are any questions relating to this Amendment or to the application in 
general, it would be appreciated if the Examiner would telephone the undersigned attorney 
concerning such questions so that the prosecution of this application may be expedited. 

Please charge any shortage or credit any overpayment of fees to BLANK ROME LLP, 
Deposit Account No. 23-2185 (123593.00106). In the event that a petition for an extension of 
time is required to be submitted herewith and in the event that a separate petition does not 
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accompany this response, Applicants hereby petition under 37 CFR 1.136(a) for an extension of 
time for as many months as are required to render this submission timely. Any fee due is 
authorized above. 

Respectfully submitted, 
BLANK ROME LLP 



By: /Peter S. Weissman/ 
Peter S. Weissman 
Reg. No. 40,220 

600 New Hampshire Ave., N. W. 
Washington, D.C. 20037 
Telephone: (202) 944-3000 
Atty. Docket No.: 123593-00106 
Date: May 18, 2010 
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